🧠 Brain-Controlled Kitchen 🍳

A seminar project on Brain-Computer Interfaces.

Check out the video of the BCI interaction

🧠 About the project

Cooking can be both the most fun and the most annoying activity. Think of the time when you wanted to try a new recipe: your hands are covered with dough and oil, your phone screen keeps switching off, you forget what the next step was, and, while you are washing your hands and trying to scroll through the recipe, the sauce in your pan starts boiling and vegetables in the oven get overcooked...

Multitasking can be truly challenging, and that's why DFKI has built a smart kitchen with a cooking assistant Kochbot, which helps you navigate through the recipe. You interact with Kochbot via speech, and it tells you the next step in the recipe, so you can focus on cooking directly. Sounds good, right?

But we went a step further and asked ourselves: what if you could navigate through the recipe just with your thoughts?

💬 Interaction plan

We had in our posession:

A Smart kitchen consisting of a hob, a sink and an extractor hood;
An EEG cap for the user to wear;
A recipe of pasta pesto (directly from Italy) with the instructions;
A screen to display the instruction steps.

Smart kitchen

We hard-coded the recipe and the instruction steps. Our instruction interacts directly with the kitchen via a software, controlling induction hob, water in the sink, and an extractor hood. The idea was to allow the user to navigate through the recipe using Imagined Speech. The user has to think of some command, then our classifer recognizes and executes it.

Three commands are available for the user: next, back, and repeat.

Next: proceed to the next step of the instruction.
Repeat: repeat the current step.
Back: go one step back in the instruction.

Smart kitchen

Data collection

First, we recorded EEG signals from 3 members of our group using a non-wireless 24-channel EEG system with ANT Recorder (sampling rate = 500 Hz);
Each participant was presented with instruction words "next", "back", "repeat" in blocks of 10. Each block was randomly presented 10 times.
We trained a classifier for each participant and performed 10-fold cross-validation on heldout data. Then we proceeded with the participant for whom the classifier's accuracy was the highest.

There are several ways how Brain-Computer Interfaces can facilitate cooking exprience:

It can be very useful for people with disabilities;
It is often noisy in the kitchen, so voice commands can be difficult to understand. Imagined Speech is a better alternative

🧠 Brain-Controlled Kitchen 🍳

Check out the video of the BCI interaction

Check out the video of the BCI interaction

🧠 About the project

💬 Interaction plan

📚 Methodology

Data collection

📐 Why use Brain-Computer Interfaces?